CI: Add Roborazzi Screenshot comparison GitHub Action#20951
Open
david-allison wants to merge 5 commits intoankidroid:mainfrom
Open
CI: Add Roborazzi Screenshot comparison GitHub Action#20951david-allison wants to merge 5 commits intoankidroid:mainfrom
david-allison wants to merge 5 commits intoankidroid:mainfrom
Conversation
david-allison
force-pushed
the
roborazzi-ci-1
branch
from
2a81005 to
d8daceb
Compare
This comment was marked as resolved.
This comment was marked as resolved.
david-allison
force-pushed
the
roborazzi-ci-1
branch
from
d8daceb to
e44b566
Compare
Using Roborazzi for the implementation * 'screenshot_store': runs on `main` store an artifact with a baseline * 'screenshot_compare': an artifact is generated on PRs if changes are , detected against `main`'s artifact * 'screenshot_comment': if an artifact is generated, a PR comment is added Original sources https://github.com/takahirom/roborazzi/blob/main/.github/workflows/CompareScreenshot.yml https://github.com/takahirom/roborazzi/blob/main/.github/workflows/CompareScreenshotComment.yml https://github.com/takahirom/roborazzi/blob/main/.github/workflows/StoreScreenshot.yml - compare_screenshot_comment: quote the bot email so the [bot] brackets aren't interpreted by the shell - StudyScreenScreenshotTest: keep per-test-class subdirectory in the capture path so future screenshot tests don't collide - Use dedicated Roborazzi gradle tasks (compareRoborazziPlayDebug) - Remove pinned workflows to match repo style - Use open source gradle cache provider - Rename workflows - Add Apache header Issue 20942 Assisted-by: Claude Opus 4.7
A branch was deemed too risky from a security perspective Assisted-by: Claude Opus 4.7
* remove the nested folder tree (build/outputs/roborazzi) * name the directory: screenshot-diff.zip contains `screenshot-diff-pr-123` so conflicts don't occur and cleanup is easier Assisted-by: Claude Opus 4.7 - all Issue 20942
If the regressions were fixed, mark the PR as resolved Do not post a comment if one already existed Issue 20942 Assisted-by: Claude Opus 4.7 - all
Store only needs to be executed on `main` We run on some file changes, in case they break the upload process, but this is only defensive. Note: this does upload artifacts, this could be avoided, but the complexity makes the 'happy path' of this workflow harder to reason about. Issue 20942 Assisted-by: Claude Opus 4.7
david-allison
force-pushed
the
roborazzi-ci-1
branch
from
e44b566 to
811c380
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Assisted-by: Claude Opus 4.7
Purpose / Description
Brayan wanted to get our Roborazzi screenshot tests running in CI, so a user was informed explicitly if their code changes changed the app visuals.
This was primarily for his theming/Material You improvements, but this affects all users positively, allowing us to automate regression testing, or allowing a user to add a test to produce up-to-date screenshots exposing a new feature across all themes.
Fixes
Approach
Follow the Roborazzi guide, then make changes
This introduces 3 files:
screenshot_store.yml: store a baseline in an artifact for commits tomainscreenshot_compare.yml: on a PR branch, build a snapshot and compare it withmainscreenshot_comment.yml: whenscreenshot_compare.ymlcompletes, add a comment to the PR listing changes, OR updates the comment if changes are resolvedMajor Changes
ankidroid/Anki-Android, reference the artifactscreenshot-diffartifact is now more useablescreenshot-diff-pr-Nx.pngx_actual.pngx_compare.pngHow Has This Been Tested?
Checklist